Text Database Discovery Using Neural Net Agent

As the number and diversity of text databases on the Internet increases rapidly, users are faced with finding the text databases that are relevant to the user query. Identifying the relevant text databases out of many candidates for a given query is called the text database discovery problem. In this paper, we propose a neural net based approach to the text database discovery problem. First, we present a neural net agent that learns about underlying text databases from the user’s relevance feedback. For a given query, the neural net agent, which is sufficiently trained on the basis of the BPN learning mechanism, discovers the text databases associated with the relevant documents and retrieves those documents effectively. In order to scale our approach with the large number of text databases, we also propose the hierarchical organization of neural net agents which reduces the total training cost at the acceptable level. Finally, we evaluate the performance of our approach by comparing it to those of the conventional wellknown statistical approaches.